MDEV-35000 dict_table_close() is a performance hog #3729
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
dr-m
commented
Jan 21, 2025
Comment on lines
1119
to
1123
| /** Deinitialize InnoDB statistics when closing the last table handle. */ | ||
| class InnoDB_share final : public Handler_share | ||
| { | ||
| /** InnoDB table that contains persistent statistics */ | ||
| dict_table_t &table; |
There was a problem hiding this comment.
The data member needs to be a pointer and not a reference. On table-rebuilding operations or when a DROP TABLE operation is committed, we must update this to point to the new table or invalidate this to nullptr. Currently, there are plenty of heap-use-after-free errors in table-rebuilding tests.
innodb_stats_transient_sample_pages, innodb_stats_persistent_sample_pages: Change the type to UNSIGNED, because the number of pages in a table is limited to 32 bits by the InnoDB file formats. btr_get_size_and_reserved(), fseg_get_n_frag_pages(), fseg_n_reserved_pages_low(), fseg_n_reserved_pages(): Return uint32_t. The file format limits page numbers to 32 bits. dict_table_t::stat: An Atomic_relaxed<uint32_t> that combines a number of metadata fields. innodb_copy_stat_flags(): Copy the statistics flags from TABLE_SHARE or HA_CREATE_INFO. dict_table_t::stats_initialized(), dict_table_t::stats_is_persistent(): Accessors to dict_table_t::stat.
dict_table_close(table): Replaced with table->release(). dict_table_close(table, thd, mdl): Remove the parameter "dict_locked". Do not try to invalidate the statistics. InnoDB_share::~InnoDB_share(): Deinitialize the statistics when the last table handle is closed. ha_innobase::innodb_share_register(): Register InnoDB_share(table) so that the statistics will be deinitialized after the last ha_innobase::close(). ha_innobase::statistics_init(): Replaces dict_stats_init(table). FIXME: Properly detect Partition_share and replace set_ha_share_ptr(nullptr) with something that accesses the InnoDB_share for the current (sub)partition. FIXME: Incorrect result innodb.innodb_stats_auto_recalc_on_nonexistent FIXME/TBD: Changed result innodb.innodb-index-online
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Every time
dict_table_close()is decrementing the table reference count to 0, it is performing some expensive logic that aims to ensure that duringFLUSH TABLES, the InnoDB persistent statistics will be deinitialized, so that during a subsequentha_innobase::open()the statistics will be reloaded. The purpose of this is to ensure thatFLUSH TABLESwill make any changes tomysql.innodb_table_statsandmysql.innodb_index_statsaccessible. This is an overkill, becauseFLUSH TABLESis executed very rarely.There is a better way to do this: Keep track of
FLUSH TABLESonly, by overloadingHandler_share::~Handler_share().This also includes some preparation to simplify the initialization and clearing of the statistics:
innodb_stats_transient_sample_pages,innodb_stats_persistent_sample_pages: Change the type toUNSIGNED, because the number of pages in a table is limited to 32 bits by the InnoDB file formats.btr_get_size_and_reserved(),fseg_get_n_frag_pages(),fseg_n_reserved_pages_low(),fseg_n_reserved_pages(): Returnuint32_t. The file format limits page numbers to 32 bits.dict_table_t::stat: AnAtomic_relaxed<uint32_t>that combines a number of metadata fields.innodb_copy_stat_flags(): Copy the statistics flags fromTABLE_SHAREorHA_CREATE_INFO.dict_table_t::stats_initialized(),dict_table_t::stats_is_persistent(): Accessors todict_table_t::stat.Release Notes
innodb_stats_transient_sample_pages,innodb_stats_persistent_sample_pages: Change the type toBIGINT UNSIGNEDtoUNSIGNED, because the number of pages in a table is limited to 32 bits by the InnoDB file formats.How can this PR be tested?
This code is rather well covered by the regression test suite.
The
FLUSH TABLESfunctionality with respect to InnoDB persistent staitistics is covered by the following tests:innodb.innodb_stats_auto_recalc_on_nonexistentinnodb.innodb_stats_fetchBasing the PR against the correct MariaDB version
mainbranch.This is a performance bug fix. The way how the statistics are initialized and updated have been largely rewritten in MariaDB Server 10.6. It is not trivial to apply these changes to 10.5, which is fairly close to its End Of Life anyway.
PR quality check